Multi-phase software delivery

ABSTRACT

A service manager component associated with a set of hosts receives an update to be implemented on the set of hosts. The service manager component can then determine a penalty model that approximates the likely impact associated with an error in the update. Based on the penalty model, the service manager component selects a first subset of hosts to receive and implement the update and an observation window to determine whether an error has emerged or has been detected. If no errors are detected during the observation window, the service manager component can select additional subsets and observations windows and repeat the process or, alternatively, implement the update in the remaining set of hosts and monitor the system until it receives the next update.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.13/076,163, entitled MULTI-PHASE SOFTWARE DELIVERY, and filed on Mar.30, 2011, the entirety of which is incorporated herein by reference.

BACKGROUND

Generally described, computing devices utilize a communication network,or a series of communication networks, to exchange data. Companies andorganizations operate computer networks that interconnect a number ofcomputing devices to support operations or provide services to thirdparties. The computing systems can be located in a single geographiclocation or located in multiple, distinct geographic locations (e.g.,interconnected via private or public communication networks).Specifically, data centers or data processing centers, herein generallyreferred to as a “data center,” may include a number of interconnectedcomputing systems to provide computing resources to users of the datacenter. The data centers may be private data centers operated on behalfof an organization or public data centers operated on behalf, or for thebenefit of, the general public. To facilitate increased utilization ofdata center resources, virtualization technologies may allow a singlephysical computing device to host one or more instances of virtualmachines that appear and operate as independent computing devices tousers of a data center. With virtualization, a single physical computingdevice can create, maintain, delete, or otherwise manage virtualmachines in a dynamic matter.

Regardless of whether virtualization technologies are utilized, users,via client computing devices, can transmit requests to computingdevices, such as computing devices at data centers, to process dataprovided by, or on behalf of, the requesting client computing device,often referred to as a “Web service” or “service.” The client computingdevices can typically request the processing of data (e.g., a “servicerequest”) through the transmission of data that has been organized inaccordance with a pre-established format, such as an ApplicationProtocol Interface (“API”). For example, a user can access various dataprocessing services via a browser-based software application hosted onthe client computing device. Based on the information included in aclient computing device service request, the receiving computing deviceprocesses the request and can return responsive data or confirm that therequested data processing has been completed. From the perspective ofthe user at the client computing device, the utilization of suchservices can provide the impression that the requested services areimplemented on the client computing device.

In a typical embodiment, one or more third party service providersmaintain various computing devices in a data center that process theclient computing device service requests, generally referred to ashosts. Periodically, the third party service provider or data centerprovider may wish to implement updates, upgrades or other modificationsto the software maintained by the hosts and utilized in conjunction withthe processing of service requests. Despite substantial testing of anyproposed updates, upgrades, or modifications, any software modificationto a service host may contain errors (e.g., “bugs”), some of which maybe latent or not fully realized until the update, upgrade ormodification has been implemented. However, the development or emergenceof a software error once the software has been implemented across afleet of hosts that provide a service can have a significant impact onthe availability of the service and the potential damage caused toclient computing devices or data associated with client computingdevices.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages will becomemore readily appreciated as the same become better understood byreference to the following detailed description, when taken inconjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrating an embodiment of a serviceprovider network for providing network based services to serviceclients;

FIG. 2A is a block diagram of service provider network of FIG. 1illustrating the transmission of updates or modifications to beimplemented by a set of hosts associated with a service provider;

FIG. 2B is a block diagram of service provider network of FIG. 1illustrating the selection of a first subset of hosts to implement anupdate or modification over a determined observation window;

FIG. 2C is a block diagram of service provider network of FIG. 1illustrating the selection of a second subset of hosts to implement anupdate or modification once the observation window has expired;

FIG. 3 is a flow diagram of an update processing routine implemented bya service manager component; and

FIG. 4 is a flow diagram of a subset and observation windowdetermination subroutine implemented by a service manager component.

DETAILED DESCRIPTION

Generally described, the present disclosure relates to theimplementation and management of software in a networked basedenvironment. More specifically, the present disclosure relates to themanagement of software that is to be implemented on multiple computingdevices in accordance with a multi-phased distribution. Illustratively,aspects of the present disclosure will be described with regard to theimplementation, configuration, or incorporation of software updates,upgrades, or other modifications, which will be generally referred to as“updates.” Additionally, aspects of the present disclosure will bedescribed with regard to the management of the updates on a set ofclient computing device hosts that correspond to service providersproviding networked-based services. Although utilized for purposes ofillustrative examples, one skilled in the art will appreciate that suchillustrative examples should not necessarily be construed as limiting.

In accordance with an illustrative embodiment, a service managercomponent associated with a set of hosts receives an update to beimplemented on the set of hosts. The service manager component can thendetermine a penalty model that approximates the likely impact associatedwith an error in the update. Based on the penalty model, the servicemanager component selects a first subset of hosts to receive andimplement the update and an observation window to determine whether anerror has emerged or has been detected. Illustratively, the selection ofthe number of hosts in the first subset and the observation window arebased on optimizations of corresponding variables in a penalty model. Ifno errors are detected during the observation window, the servicemanager component can select additional subsets and observations windowsand repeat the process or, alternatively, implement in the update in theremaining set of hosts.

FIG. 1 is a block diagram illustrating an embodiment of a serviceprovider network 100 for providing network based services. The serviceprovider network 100 includes a number of service clients 102, such asclient computing devices, that can request services from a serviceprovider. The service clients 102 can correspond to individual computingdevices, such as personal computing devices, mobile devices, servers,etc. that have been configured with software applications fortransmitting service requests, such as via one or more APIs. The serviceclients 102 can also correspond to multiple computing devices or agrouping of a set of network computing devices that transmit servicerequests. The service clients 102 transmit requests via a communicationnetwork 104, such as the Internet, intranets, local area networks,wireless networks, wan access networks, and the like. The communicationnetwork may correspond to either public or private communicationnetworks.

In communication with the service clients 102 via the communicationnetwork 104, is a service provider network 106 that is associated with aset of hosts for providing network based services responsive to serviceclient requests. As illustrated in FIG. 1, the service provider network106 includes a host manager component 108 that manages updates to a setof hosts 110. Illustratively, the set of hosts 110 can correspond tophysical computing device implementing one or more software applicationsand communicating with other computing devices for processing serviceclient requests. Additionally, at least some portion of the serviceprovider network 106 can correspond to a hosted virtual machine networkin which one or more of the set of hosts 110 are implemented as virtualmachine instances in a physical computing device. Accordingly, theservice provider network 106 should be considered as a logicalrepresentation of a set of hosts regardless of the manner in which sucha logical representation is implemented on physical computing devices.Likewise, although the service provider network 106 is illustrated asinterconnected, one skilled in the relevant art will appreciate that atleast some portion of the service provider network 106 may begeographically distributed and interconnected via a private or publiccommunication network.

Also in communication with the service provider network 106, via thecommunication network 104, are one or more service vendors orapplication providers 112. Illustratively, the service vendors 112 mayinclude third party vendors that provide the software applicationsexecuted by the hosts to provide services. The service vendors may alsocorrespond to an entity that also provides the service provider network106 and does not necessarily have to correspond to a true third party.As will be described below, the service vendor 112 can provide theupdates to be implemented on the hosts 110.

With reference now to FIGS. 2A-2C, various embodiments for themanagement of software updates on the service provider network 106 willbe described. However, one skilled in the relevant art will appreciatethat illustrative interaction and communications may include, orotherwise involve, additional components not illustrated in theillustrative drawing figures. With reference to FIG. 2A, a servicevendor 112 transmits an update to be implemented on a set of hosts 110.The service vendor 112 can illustratively provide the update, which cancorrespond to an upgrade, modification, patch, configuration, and thelike. The service vendor 112 can also transmit criteria for implementingthe update including the identification of the hosts that shouldincorporate the update, timing for completion of the update, testingverification, troubleshooting information, etc. In the event the sameentity that controls the service provider network 106 also correspondsto the service vendor 112, the illustrative interaction can occur withinthe service provider network 106 between one or more components. Theservice vendor update can correspond to software applications utilizedin conjunction with the providing of services, such as an update to adatabase server application utilized in a content management service,but that do not expressly correspond to the service itself (e.g., thecontent management service). Additionally, the service vendor update cancorrespond to software application utilized specifically for theproviding of services.

Upon receiving of the update, a host manager component 108 identifiesthe set of hosts 110 that will receive the update. In one example, thehost manager component 108 can utilize information included in thetransmission of the update that identifies one or more hosts 110 toreceive the update or provides criteria for selecting the set of hosts110. In another example, the host manager 106 can perform identify theset of hosts 110 to receive the update based on an affiliation with theservice vendor 112, an identification of software applicationscorresponding to the update, hosts associated with a particular entity,and the like.

With reference to FIG. 2B, the host manager component 108 thendetermines a first subset of hosts 110 that will receive update andcalculates an observation window for monitoring the implementation ofthe update on the first subset of hosts. Illustratively, thedetermination of the first subset of hosts 110, in terms of the numberof hosts, and the observation window can be based on analysis of anassessment of the likely impact of an error emerging from the update,generally referred to as a penalty model. More specifically, the penaltymodel can be generated based on an estimated distribution of the time inwhich an error will first occur based on the implementation or,deployment and execution, of the update in a set of hosts and theestimate time that lapses between occurrence and correction of theerror. Illustratively, the penalty model can be generic to all set ofhosts and all types of updates. Alternatively, the penalty model may becustomized to types of updates, types of hosts, types of operatingsystems, service clients 102, service vendors 112, or combinationsthereof. A more detailed discussion of the determination of the numberof hosts, and the observation window based on a penalty model will bedescribed below. Based on the calculated number of hosts and observationwindow, the host manager component 108 then causes the implementation ofthe update on the selected first subset of hosts 110.

With reference now to FIG. 2C, the host manager component 108 processesall data corresponding to the monitoring of the implementation of theupdate on the first subset of hosts 110. The observations may be activeby receiving reports (including prompting) from the first subset ofhosts 110. Additionally, the observations may be passive by waiting forreporting of any type of error associated with the first subset ofhosts. If an error is detected, the host manager component 108 canimplement mitigation techniques, such as recalling the update, restoringa previous version of the software application, reconfiguration of thefirst subset of hosts, etc. If no errors are detected during theobservation window, the host manager component 108 can then selectadditional subset of hosts for further testing or cause the deploymentand execution of the update on the remaining hosts previouslyidentified. Illustratively, the host manager component can implement anynumber of deployment and execution phases in the deployment andexecution of an update. Additionally, the host manager component 108 cancontinuously monitor the execution of an update once it has beendeployed on the set of hosts. In this regard, the final observationwindow for deployment on all the set of hosts can be considered to beequal to the remaining time until the next update will be deployed,which is generally referred to as a cycle time.

Turning now to FIG. 3, a routine 300 implemented by a component, such asa host manager component 108 (FIG. 1) for processing updates will bedescribed. Although routine 300 will be described with regard toimplementation by a host manager component 108, the routine may beimplemented by any number of components or combination of components. Atblock 302, the host manager component 108 obtains an update forimplementation on a set of hosts 110. As previously described, theupdate may be provided by service vendor 112, such as a third partyservice vendor. At block 304, the host manager component 108 determinesthe set of hosts that will receive the update. In one embodiment, theupdate may identify the set of hosts that are to receive the update. Inanother embodiment, the update may identify criteria that will beutilized by the host manager component 108 to identify the set of hosts.In a further embodiment, the host manager component 108 may identify theset of hosts independently from the receipt of the update.

At block 306, the host manager component 108 determines a first subsetof hosts to implement the update. At block 308, the host managercomponent 108 then determines an update observation window. Aspreviously described, illustratively, the host manager component 108 canidentify a penalty model that assesses a penalty based on a distributionof the time in which an error will first occur during the execution ofthe deployed update and the likely time between the emergence of anerror and the ability to remedy the error. The penalty models may begeneric to all updates for the service provider network 106.Alternatively, the penalty model may be customized to specific updates,types of updates, types of service vendors, and the like. Based on theidentified penalty model, the host manager component 108 selects valuesfor the number of hosts in the first subset and the observation windowsto minimize the assessed impact, e.g., the penalty in the penalty model.A more detailed discussion of illustrative penalty models andoptimizations will be described with regard to FIG. 4, below.

At block 310, the host manager component 108 transmits the update to theselected hosts in the first subset of hosts. At decision block 312, atest is conducted to determine whether an error has occurred during thedetermined observation window. If so, at block 314, the host managercomponent 108 processes the error and routine 300 ends. Illustratively,the host manager component 108 can implement various mitigationtechniques in the event an error is detected including restoringprevious versions of software application, instantiating new virtualinstances of a host (with previous versions of software applications),turning off features, and the like. One skilled in the relevant art willappreciate that decision block 312 may correspond to a loop that isexecuted throughout the observation window.

If no errors are detected during the observation window, at decisionblock 318, a test is conducted to determine whether the host managercomponent 108 should implement the update on additional subsets ofhosts. Illustratively, the host manager component 108 may implement theupdate in a plurality of phases in which each phase may increase thenumber of hosts that receive the update. If additional subsets of hostsare to receive the update, at block 320, the host manager component 108determines the next subset of hosts to receive the update and theroutine 300 returns to block 308. Illustratively, the next subset ofhosts can correspond to an incremental number of hosts to the firstsubset of hosts that have received and implemented the update.Alternatively, the next subset of hosts can correspond to anindependently determined subset of hosts that may or may not have anyoverlap with the first subset of hosts. Still further, in anotherembodiment, the next subset of hosts can corresponds to any remaininghosts in the set of hosts that have not yet deployed the update. In thisembodiment, the host manager component 108 can set the observationwindow determined in block 308 to be equal to, or least substantiallyclose, to the time remaining for the deployment of the update in the setof hosts (e.g., observing the set of hosts for the remainder of therelease cycle).

If no additional subsets of hosts are to receive the update, at block322, the routine 300 terminates.

With reference now to FIG. 4, a subroutine 400 for determining a numberof hosts in a subset of hosts and an observation window based on apenalty model will be described. Illustratively, subroutine 400 may beimplemented by a host manager component 108 or other componentsassociated with the service provider network 106. At block 402, the hostmanager component 108 determines the total number of hosts and a maximumamount of time to implement a target update. As previously described,the host manager component 108 can utilize criteria provided by aservice vendor 112 associated with the target update to determine thetotal number of hosts. Alternatively, the host manager component 108 canmake an independent determination of the total number of hosts.Likewise, the maximum amount of time to implement a target update maycorrespond to defaults set by the service provider network 106, theservice vendor 106, service level agreements or in accordance with anestablished scheduled for implementing updates.

At block 404, the host manager component 108 determines a distributionof time to an occurrence of a first failure for the target update.Illustratively, the distribution of time to a first failure correspondsto an approximation of the time in which a failure based on execution ofa deployed update will occur. Illustratively, the distribution of timeto a first failure can be represented as an exponential distribution.The distribution of time to a first failure may be generic to allservice providers or based, at least in part, on specific serviceproviders, service vendors, types of updates, types of services, etc.The distribution of time to a first failure may also be based onhistorical data. At block 406, the host manager component 108 determinestime parameters associated with a time between emergence of an error andthe mitigation of the error. As previously described, the host managercomponent 108 may cause the implementation of various mitigationtechniques that may depend on the type of error that may have emerged orthe type of update involved. Depending on the complexity of themitigation technique, there may be a specific amount of time in which anerror is present (and perhaps known), but cannot yet be mitigated.During this time, the “penalty” is incurred by the service provider.

At block 408, the host manager component 108 calculates a penalty modelbased on the total number of hosts, distribution of time to a firstfailure and time parameters. Illustratively, the penalty model cancorrespond to assessment of the impact if an error is found duringexecution of the update on a first subset of hosts within the currentobservation window and if an error is found during execution of theupdate on a larger subset of hosts within the next observation window.The below equation corresponds to an illustrative penalty model for atwo-phase deployment:

P(n,t)=n·λ·D·(1−e ^(−n·λ·t))+e ^(−n·λ·t) ·N·λ·D·(1−e ^(−n·λ·(C−t)))

-   -   where 0<n≦N and 0<D≦t≦T<<C    -   and where    -   P: assessed penalty for errors in an update    -   n: number of hosts to be selected in the first subset of hosts;    -   t: observation window/delay (e.g., a current observation window)    -   λ: rate parameter of an exponential distribution modeling time        to first failures; 1/λ is the expected time to first failures    -   D: time interval between error occurrence and error detection        and removal N: total number of hosts    -   T: upper bound for current observation window    -   C: time for next scheduled update (e.g., the release cycle)

At block 410, the host manager component determines values for thenumber of hosts to be selected in the first subset of hosts andobservation window based on the penalty model. As illustrated above, theillustrative penalty model factors in a penalty for an error occurringduring execution of the update on a first subset, “n,” within the firstobservation window namely, n·λ·D·(1−e^(−n·λ·t)). The illustrativepenalty model also factors in a penalty for an error occurring duringexecution of the update on all the hosts within the second observationwindow (whose size is the release cycle minus the size of the firstobservation window), namely, e^(−n·λ·t)·N·λ·D·(1−e^(−N·λ·(C−t))).Accordingly, the host manager component attempts to optimize the penaltymodel such that the penalty for an error occurring during implementationin all the hosts is minimal. In one embodiment, the optimization of thepenalty model is achieved via selection of different values for numberhosts, “n,” and the observation window, “t.”

Illustratively, the host manager component 108 can implement theoptimization process by selecting a value for “n” to minimize thepenalty model for any given value of t. Specifically, the host managercomponent 108 can utilize the first and second partial derivatives ofthe penalty model equation with respect to “n” to model the change inpenalty model. Illustratively, based on a single root of the firstpartial derivative, the host manager component 108 can determine anoptimal “n” having a value between “0” (e.g., indicative of an exclusivesubset from the set of hosts) and “N” (e.g., indicative of an inclusivesubset including all the set of hosts). Additionally, “T” is always anoptimal value of “t” for this model. Still further, when “N” is anoptimal value of “n,” the two-phase deployment degenerates to one-phasedeployment, and the two observation windows merge into a singleobservation window whose size is “C,” the release cycle.

At block 412, the subroutine 400 terminates with the identification ofthe optimal values for n and t.

It will be appreciated by those skilled in the art and others that allof the functions described in this disclosure may be embodied insoftware executed by one or more processors of the disclosed componentsand mobile communication devices. The software may be persistentlystored in any type of non-volatile storage.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements, and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art. It willfurther be appreciated that the data and/or components described abovemay be stored on a computer-readable medium and loaded into memory ofthe computing device using a drive mechanism associated with a computerreadable storing the computer executable components such as a CD-ROM,DVD-ROM, or network interface further, the component and/or data can beincluded in a single device or distributed in any manner. Accordingly,general purpose computing devices may be configured to implement theprocesses, algorithms, and methodology of the present disclosure withthe processing and/or execution of the various data and/or componentsdescribed above.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

What is claimed is:
 1. A method for managing updates to a set of hostsassociated with a service provider, comprising: obtaining an update tobe implemented on a set of computing devices; obtaining acharacterization of a numerical measure that quantifies an erroroccurring during deployment of the update in a first subset of the setof computing devices and an error occurring during deployment of theupdate in the set of computing devices; determining a size of the firstsubset based, at least in part, on a minimization of the numericalmeasure; causing a deployment of the update in the first subset ofcomputing devices corresponding to the determined size of the firstsubset; and determining a timing for a deployment of the update in theset of computing devices based, at least in part, on an attributeassociated with the deployment of the update in the first subset ofcomputing devices; the method performed programmatically by one or morecomputing systems under control of executable program code.
 2. Themethod of claim 1, wherein the update corresponds to at least one of anupgrade, modification, patch, or configuration.
 3. The method of claim1, wherein the numerical measure includes an amount of time in which atleast one error associated with the deployment of the update is presentand not yet mitigated.
 4. The method of claim 1 further comprisingdetermining a duration of time associated with the deployment of theupdate in the first subset.
 5. The method of claim 4, whereindetermining the duration of time comprises determining the duration oftime based, at least in part, on a minimization of the numericalmeasure.
 6. The method of claim 4, wherein determining the duration oftime comprises determining the duration of time based, at least in part,on the size of the first subset.
 7. The method of claim 1, wherein thecharacterization of the numerical measure includes at least onemathematical formula for the numerical measure.
 8. The method of claim7, wherein the at least one mathematical formula is based, at least inpart, on a distribution of time before an initial error occurs duringthe deployment of the update.
 9. The method of claim 7, wherein theminimization of the numerical measure is based, at least in part, onpartial derivatives applicable to the at least one mathematical formula.10. The method of claim 1, wherein the attribute associated with thedeployment of the update in the first subset corresponds to a timing ofdetecting an error during the deployment of the update in the firstsubset.
 11. A system for managing updates comprising: a set of hosts forproviding services to service clients, the set of hosts each including aprocessor and maintaining software applications; a host managercomponent, implemented on a computing system including one or moreprocessors and memory, the host manager component operative to: obtainan update to be deployed on the set of hosts; determine a size of afirst subset of the set of hosts based, at least in part, on aminimization of a first quantification of an error occurring duringdeployment of the update in the first subset of hosts and a secondquantification of an error occurring during deployment of the update inthe set of hosts; cause the deployment of the update in the first subsetof hosts corresponding to the determined size of the first subset; anddetermine a timing for a deployment of the update in the set of hostsbased, at least in part, on an attribute associated with the deploymentof the update in the first subset of hosts.
 12. The system of claim 11,wherein the attribute associated with the deployment of the update inthe first subset includes an amount of time in which no error isdetected during the deployment of the update in the first subset. 13.The system of claim 11, wherein the host manager component is furtheroperative to obtain at least one of a criterion for identifying the setof hosts, timing for completion of the update, or information fortesting verification.
 14. The system of claim 11, wherein the hostmanager component is further operative to detect an error during thedeployment of the update in the first subset of hosts.
 15. The system ofclaim 14, wherein the host manage component is further operative tocause mitigation of the detected error.
 16. The system of claim 15,wherein mitigation of the detected error includes at least one ofrecalling the update, restoring a previous version of softwareapplication, instantiating a new virtual instance of a host, turning offa feature, or reconfiguration of the first subset of hosts.
 17. Thesystem of claim 11, wherein the first and second quantifications arecharacterized in accordance with a penalty model.
 18. A non-transitorycomputer readable storage medium storing computer executableinstructions that instruct one or more processors to perform operationscomprising: obtaining an update to be deployed on a set of computingdevices; determining a size of a first subset of the set of computingdevices based, at least in part, on a minimization of a numericalmeasure that quantifies an error occurring during deployment of theupdate in the first subset of computing devices and an error occurringduring deployment of the update in the set of computing devices inaccordance with a characterization of the numerical measure; causing adeployment of the update in the first subset of computing devicescorresponding to the determined size of the first subset; and causing adeployment of the update in the set of computing devices based, at leastin part, on a timing of error occurrence during the deployment of theupdate in the first subset of computing devices.
 19. The non-transitorycomputer readable storage medium of claim 18, wherein thecharacterization of the numerical measure includes at least one of aduration of time associated with the deployment of the update in thefirst subset, a time interval between error occurrence and errormitigation, or a time for next scheduled update.
 20. The non-transitorycomputer readable storage medium of claim 18, wherein the operationsfurther comprise: determining a size of a second subset of the set ofcomputing devices; and causing a deployment of the update in the secondsubset prior to the deployment of the update in the set of computingdevices.
 21. The non-transitory computer readable storage medium ofclaim 20, wherein the size of the second subset is larger than the sizeof the first subset.